Skip to content

Deployment#8

Draft
tomaszslabon wants to merge 167 commits intomasterfrom
deployment
Draft

Deployment#8
tomaszslabon wants to merge 167 commits intomasterfrom
deployment

Conversation

@tomaszslabon
Copy link
Copy Markdown

No description provided.

prestwich and others added 30 commits September 19, 2019 09:15
feature: rewrite some getters and setters
WIP: Adding Getters and Setters in keeper.go
Shadowfiend and others added 30 commits May 20, 2020 20:01
Ideally the relay doesn't pile up unconfirmed transactions at all. Count
ticks so that, as a transaction sits for longer, it is replaced by
gradually increased gas costs.

Also update default gas price to 20, which is more reasonable on
mainnet, and adjust the curve to increase by 20% increments instead of
by full DEFAULT_GAS_PRICE increments.
It's no longer needed, as the transaction nonce of the transaction under
consideration is the one used to make decisions about gas price, rather
than the overall latest completed transaction.
tick information is passed along with each new transaction submission,
and tx results are tracked all the way down. Finally, "known
transaction" errors are caught and retried with a higher gas price after
a wait.

Still missing: handling _underpriced_ transactions.
In cases where a transaction fails to go through, some logging clarifies
the nonce and gas price information.
Since underpriced transactions indicate the submitted transaction for
that nonce has too low a gas price, entering the gas-bump loop allows
the gas bumping to get the transaction to an appropriate price.

To do this, _track_tx_result can now operate without a tx_id. This is
because, in the case of underpriced transactions, no tx_id is available
to pass along. A lack of tx_id is treated as a transaction needing
retry.
Resubmissions with higher gas prices were not being tracked, so only one
gas bump could occur. These are now properly tracked for success, so
that they can have their gas price bumped if they don't succeed in a
reasonable amount of time.
Instead, wait 60 seconds and then try again. This is normal when gas
prices require multiple bumps to clear the related transactions.
Before, there was an equivalent gas price bump for remaining unmined for
30 seconds and for having additional transactions stuck behind it. Now,
having additional transactions stuck behind a transaction gives it a
much higher gas bump (2^n instead of just n), ensuring that transactions
that are blocking the contract clear quickly.
This number was actually mined_tx_count + 1, apparently. This also lets
us drop an if statement, simplifying code flow.
Additionally, this scenario is logged at WARN level so there's a trace
that the maintainer is boosting gas in this case.
This typically happens because a lower-priced version of the same
transaction cleared while a higher-priced version was about to be
submitted. Rather than bomb out, the transaction submission is simply
skipped in that case.
This makes for easier correlation of logs during as boosts when multiple
transactions are in the process of retrying.
This makes a new transaction blocked ahead of this one boost the impact
of the number of ticks this transaction has been waiting.
Before, boosting gas would wait 30s at each tick, even if there was
already a tx at this nonce and this or higher gas price. This wait is
now gone, so if there's a catch-up to be done it happens in a tight
loop.
This shouldn't have much of an effect in practice, but is a clearer
expression of the intent here.
When the maintainer restarts, its transaction resubmissions start at 1
tick and have to catch up to the number of ticks of the existing pending
transactions on chain, if any. In these cases, the new gas price can
end up below the latest gas price. For these situations, the gas price
is now boosted until it either reaches MAX_GAS_PRICE or exceeds the
latest gas price.

Additionally, irrespective of how the gas bump path is handled, the path
through it is logged.
Unfortunately, this is a necessity in the current gas price climate.
Gas Attack: Add basic gas price management

The basic mechanism now is:
- We track this transaction's nonce vs the highest pending transaction nonce.
- We also track the number of ticks a transaction has been waiting (each tick is 30s).
- We multiple ticks by the number of transactions ahead of this one, and then apply
a 20% factor increase for unit from that multiplication. This has the effect of significantly 
boosting the gas price every time a new transaction is blocked by the current one.

To handle reboots, error handling has changed so that if we get a transaction underpriced
or already known error, we boost the gas price until we don't see this. That allows a newly
booted instance that's resubmitting older transactions to reach the same state as the previous
instance. And finally, to handle instances that are catching up to latest chain state, failing to find
an LCA after 5 attemps (which are time-based rather than tx-based right now) doesn't crash;
instead, it retries after a 60s pause.

Finally, max gas is now 120 gwei.
Gas is expensive enough day-to-day at this point that starting at 100
gwei makes sense. The upper limit is now above most observed gas levels
to give some headroom as well.
Right now the wait unit is the product of how many pending transactions
are ahead of this one (generally only revelevant during relay catch-up)
and the number of ticks this transaction has gone unmined (with one tick
= 30 seconds). We were cranking up by 20% of this wait unit multipled by
default gas every tick, but this commit adjusts it to crank up by 50% of
the wait unit * default gas. For a linear wait unit increase, which is
the standard expected behavior, this yields:

  Time  | Gas Price
--------------------
   0    | 100 gwei
  30s   | 150 gwei
  60s   | 200 gwei
 1m30s  | 250 gwei
  2m    | 350 gwei
 2m30s  | 400 gwei
  3m    | 450 gwei
 3m30s  | 500 gwei
  4m    | 550 gwei

With recent gas prices, this should mean relay information should land
within 4-5 minutes when gas prices are high, but much faster and cheaper
when prices are lower. If this feels too long or gas prices become more
volatile still, the numbers can be adjusted.
Molotov Cocktail: Light gas prices on fire and throw them into the building

Right now the wait unit is the product of how many pending transactions
are ahead of this one (generally only revelevant during relay catch-up)
and the number of ticks this transaction has gone unmined (with one tick
= 30 seconds). We were cranking up by 20% of this wait unit multipled by
default gas every tick, but this commit adjusts it to crank up by 50% of
the wait unit * default gas. For a linear wait unit increase, which is
the standard expected behavior, this yields:

  Time  | Gas Price
--------------------
   0    | 100 gwei
  30s   | 150 gwei
  60s   | 200 gwei
 1m30s  | 250 gwei
  2m    | 350 gwei
 2m30s  | 400 gwei
  3m    | 450 gwei
 3m30s  | 500 gwei
  4m    | 550 gwei

With recent gas prices, this should mean relay information should land
within 4-5 minutes when gas prices are high, but much faster and cheaper
when prices are lower. If this feels too long or gas prices become more
volatile still, the numbers can be adjusted.
…-maintainer-locally

added local_test configuration for contracts
…elay_maintainer

Add docs for local setup of relay maintainer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants